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ON THE TRACE OF HECKE OPERATORS FOR 
MAASS FORMS FOR CONGRUENCE SUBGROUPS II 



XiAN-JiN Ll 

Abstract. Let Ex be the Hilbert space spanned by the eigenfunctions of the non- 
Euchdean Laplacian associated with a positive discrete eigenvalue A. In this paper, 
the trace of Hecke operators Tn acting on the space E\ is computed for Hecke con- 
gruence subgroups To{N) of non-square free level. This extends the computation of 
Conrey-Li [2], where only Hecke congruence subgroups ro(A'') of square free level N 
were considered. 



H. 

|7| ■ 1. Introduction 



Let A^ > 1 be an integer, which is not square free. Denote by ro(A^) the Hecke 
congruence subgroup of level N. The non-Euclidean Laplacian A on the upper 
half-plane H is given by 






^ ■ ydx"^ dy 

00 ■ Let D be the fundamental domain of Fq ( A^) . Eigenfunctions of the discrete spectrum 

O I of A are nonzero real-analytic solutions of the equation 

O 

o 

-(— > 



such that i^i^jz) = i/j{z) for all 7 in ro(A^) and such that 



\t(j{z)\^dz < 00 

D 



H i where dz represents the Poincare measure of the upper half-plane. 

Let a be a cusp of ro(A^). We denote its stabilizer by F^. An element aa in 
PSL2{M.) exists such that aaoo = a and a~^racra = Foo- If V^ is an eigenfunction of 
the Laplacian associated with a positive discrete eigenvalue A, then it has a Fourier 
expansion [6] 

^Piaaz) = VyYl Pa(m)K,.(27r|m|y)e2— 
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at every cusp a of ro(A^), where k — y^X^^T/l and K^{y) is given by the formula 
§6.32, [18] 

(1.1) AVM^^^^'-'+^'f -""'.dt 

The Hecke operators T^, n = 1,2, •■■, (n^N) = 1, which act in the space of 
automorphic functions with respect to ro(A^), are defined by 

^ ad=n,0<b<d ^ 

An orthonornial system of eigenfunctions of A exists [6] such that each of them is an 
eigenfunction of aU the Hecke operators. We caU these eigenfunctions Maass-Hecke 
eigenfunctions. Let Aj, j = 1, 2, ■ ■ ■ , be an enumeration in increasing order of aU 
positive discrete eigenvalues of A for ro(A^) with an eigenvalue of multiplicity m 
appearing m times, and let Kj = ^J\j — 1/4. liipj{z) is a Maass-Hecke eigenfunction 
of A associated with the jth eigenvalue Aj, then 

{Tn^j){z) =Tj{n)ijj{z) 

where Pjooirn) = pjoo{d)Tj{n) if m = dn with n ^ 1, (n, dN) = 1. 

Let Ex be a Hilbert space of functions spanned by the eigenfunctions of A as- 
sociated with a positive discrete eigenvalue A. The Petersson inner product of the 
space is given by 

(1.2) {F{z),G{z))= f F{z)G{z)dz. 

Jd 

The analogue for Maass forms of Eichler-Selberg's trace formula [12], p. 85 is ob- 
tained for congruence subgroups ro(A^) of square free level A^ in Conrey-Li [2]. In 
this paper, the trace tr\Tn of Hecke operators acting on the space E\ is computed 
for congruence subgroups ro(A^) of non-square free A^. Some of this computation 
is implicit in Hejhal [4] [5] . 

Denote by hd the class number of indefinite rational quadratic forms with dis- 
criminant d. Let 

/. ON Vo + Uoy/d 
(1-3) ed = , 

where the pair {vq, uq) is the fundamental solution [10] of Pell's equation v'^ — dv? = 
4. Denote by fl the set of all the positive integers d such that (i = or 1 (mod 4) 
and such that d is not a square of an integer. Let /u(n) be the Mobius function. 
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Theorem 1. Let N be a non-square free positive integer, and let n be a positive 
integer with {n,N) = 1. If 

k\N m|f n\k den u p2l\(d,f) ^ / \ I 

for Res > 1, where the sum on u is over all the positive integers u such that 
V4n+~dkHt^ G Z and the product on p^^ runs over all distinct primes p with p^' 
being the greatest even p-power factor of {d, ^), then Ln{s) is analytic for Res > 1 
and is extended by analytic continuation to the half-plane Res > except for a 
possible pole at s = 1/2 and for possible simple poles at s = 1, | ± iKj , j = 1, 2, • • ■ . 
Moreover, we have 

trx^Tn = 2n^'^^ ReSs=i/2+i^jLn{s) 

for j = 1,2,---. 

The paper is organized as follows. In section 2, we recall the Selberg trace 
formula; see Hejhal [4] [5] and Selberg [12]. Then in section 3 we compute contri- 
butions of identity, elliptic, hyperbolic and parabolic elements to the trace formula. 
A technical part of this section is to compute the Petersson inner product of an 
Eisenstein series for ro(A^) with the image of the Eisenstein series under the action 
of Hecke operators. The result is given in Theorem 3.11. By using the Selberg trace 
formula, we obtain analyticity information about a series formed by contributions 
to the trace formula of hyperbolic elements whose fixed points are not cusps. A 
precise statement is given in Theorem 3.13. In section 4, we compute explicitly the 
total contribution to the trace formula of hyperbolic elements whose fixed points 
are not cusps. The result is stated in Theorem 4.6. Finally, Theorem 1 follows from 
Theorem 3.13 and Theorem 4.6. 

Most part of this paper was written at the American Institute of Mathematics, 
California. The author wishes to thank AIM and Brian Conrey for the support. He 
also wishes to thank Dennis Hejhal and Henryk Iwaniec for helpful conversations 
and for their encouragement for the proof of Theorem 3.11. 

2. The Selberg trace formula 
Assume that s is a complex number with Res > 1. Let 



and 



for z = x + iy and z' = x' + iy' in the upper half-plane. The kernel function k{z, z') 
is of (a)-(b) type in the sense of Selberg [12], p. 60. Let 

giu) = f k{t) 



m-- 


= (1 + ^)- 


z) = 


-,.(---'' 




\ yy 



\/t — w 
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with w = e" + e"" - 2. If 

/oo 
-oo 

then 

(2.1) <7(^)=c(l + |)^-^ 

where c = 2i/7rr(s — |)r~^(s). Since 

A'^is - -) 

[s - ^y + H 

where A{r, s) is finite for \^r\ ^ 1/2 and for Res > 0, we find that 



(2.2) hm {s---^K)h{r) = [^'"^^^'^^i^r for r - ±«; 
s^i/2+iK 2 [ 0, for r ^ ±k. 

Assume that n is a positive integer with (n, A^) = 1. Let 

a<h<d^n \U a J 

Then F* satisfies aU the requirements given in [12], p. 69. Let ai, 02, ■ ■ ■ , Oku{N) be 
a complete set of inequivalent cusps of ro(A^). We choose aa^ G P5'L2(ffi) such 
that aa^oo = o^ and a~^VaiCra^ = Too for i = 1, 2, ■ ■ ■ ,z/(A^). Let £'^(2, s) be the 
Eisenstein series for the cusp a^, let 

K{z,z')= J2 Hz,Tz'), 
Ter* 

and let 

•^(N) I r°^ az + b 1 1 

H{z,z')=Y^ J2 -^ I h{r)E^{—^,- + ir)E,{z',--ir)dr. 

i=l ad=n,0<b<d ^ "'-°o 

It follows from (2.14) of [12], the argument of [7], pp. 96-98, Theorem 5.3.3 of [7], 
and the spectral decomposition formula (5.3.12) of [7] that 

00 „ 

(2.3) d{n)h{-'-) + v^5^ /i(/.,)trA^T„ = / {K{z, z) - H{z, z)]dz 

j=i Jd 

for Re s > 1, where d{n) is the sum of positive divisors of n. 
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3. Evaluation of components of the trace 

For every element T of F*, we denote by Ft the set of all the elements of Fo(A^) 
which commute with T. Let Dt = TtXH. Elements of F* can be classified as 
the identity, the hyperbolic, the elliptic, and the parabolic elements. If T is not a 
parabolic element, we put 



c{T) = / k{z,Tz)dz. 

J Dt 



Proofs for the following lemmas from Lemma 3.1 to Lemma 3.8 are minor modi- 
fications of corresponding lemmas in Conrey-Li [2] . For the convenience of readers, 
we sketch the proofs here. 

3.1. The identity component. 

If F* contains the identity element /, then 

c(/) = [ dz. 

Jro{N)\n 

3.2. Elliptic components. 

There are only a finite number of elliptic conjugacy classes. 

Lemma 3.1. Let R be an elliptic element ofV*. Then 

c{R) = ^^ r -^j^=dt, 

2msme Jo ^/t + 4^^ 

where m is the order of a primitive element of Tr and where 9 is defined by the 
formula trace{R) = 2 cos 6*. 

Proof. Since R is an elliptic element of F*, an element a G PS'L(2,M) exists such 

that 

„ _i /cos 6* — sin6'\ ~ 
aRa = \ = R 

y sm cos J 

for some number < ^ < tt. If ((TFo(A^)cr~^)^ is the set of all the elements of 
aro(N)a~^ commuting with R, then we have 

c{R) = / k{z,Rz)dz 

where D^ = {aVo{N)a-^)^\H. 

If 7 is an element of Fq ( A^) having the same fixed points as R, then 7 commutes 
with R. By Proposition 1.16 of [13], a primitive elliptic element 70 of Fo(A^) exists 
such that (?7F?7~^)^ is generated by rjjorj~^. Since ?77o?7~^ commutes with R, it is 
of the form 

cos 6*0 — sin 6*0 

sin 6*0 cos 6*0 
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for some real number Oq. By Proposition 1.16 of [13], ^o = 'n'/m for some positive 
integer m. It follows from the argument of [7], p. 99 that 

1 r°^ r^ /I 2 I i|2 \ 

c(R) = - / ki ' , ' sin^ 9 dz. 

mJo J_^ V r J 

By the argument of [7], p. 100 we have 

TT r°^ kit) , ^ 

ciR) = / , ^ ^ -.dt. D 

2m sine J Q Jt + A sin^ 9 



3.3. Hyperbolic components. 

Let P be a hyperbolic element of T* . Then an element p exists in 5'L2(M) such 
that 



^^^--{'^ ;,0^^ 



with \p > 1. The number Ap is called the norm of P, and is denoted by NP. It 
follows that 

c(P) = I k{z,NPz)dz 



'^P 



where Dp = {prQ{N)p~^)p\H. Let Pq be a primitive hyperbolic element of SL2{1'), 
which generates the group of all elements of 5'L2(Z) commutating with P. Then 
there exists a "primitive" hyperbolic element Pi G ro(A^), which generates Fp, such 
that Pi is the smallest positive integer power of Pq among all the generators of Fp 

mro(iV). 

Lemma 3.2. Let P be a hyperbolic element ofV* such that Tp ^ {12}- ^ P\ is a 
"primitive" hyperbolic element o/ro(A^) which generates the group Tp, then 

In MP 

Proof. Since pPip~^ commutes with P, it is of the form 

Ap, 

x-pl 

for some number Ap, > 1, and hence 
The stated identity then follows. D 
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Lemma 3.3. Let P = -k^ \ n n) ^^ ^ hyperbolic element of T* such that Tp = 
{I2}. Then fixed points of P are cusps ofro{N). Moreover, Tp = {I2} if and only 
if A + D = ^{m + 1^) for some divisor m of An with m 7^ 2y/n for C 7^ 0, and if 
and only if A^ D for C = 0. 

Proof. If the fixed points of P are not rational numbers or infinity, then they are 
zeros of an irreducible polynomial ax^ + 6x + c with a = mod(A^). If (i = 6^ — 4ac 
and (w, v) is a solution of Pell's equation v'^ — du^ = 4, then 

^ -cu 
an ^ 

belongs to ro(A^) and commutes with P. This contradicts to Tp = {12}- Hence, 
fixed points of P are cusps of ro(A^). 

Conversely, if T is an element of F* having two distinct fixed points with at 
least one of them being a rational number, then we must have Ft = {12}- Hence, 

Tp = {I2} for an element P = -^ f „ j G F* if and only if the fixed points of 

P are rationals or the infinity, that is, if and only ii A + D = i(m + ^) for some 
divisor m of 4n with m 7^ 2y^ if C 7^ or A 7^ D if C = 0. D 

Every cusp of Fq ( A^) is equivalent to one of the following inequivalent cusps 

(3.1) — with UjW > 0, in, w) = 1, w\N. 

w 

Two such cusps u/w and ui/wi are Fo(A^) -equivalent if and only \i w = wi and 
u = ui modulo {w,N/w). Let a = u/w be given as in (3.1). By (2.2) and (2.3) of 
Deshouillers and Iwaniec [3], we have 

(3.2) Ta = < ^ / : c = (mod w ,iV) 

I V c 1 — cu w I ^ ^ 



and 



CTaOO = a and a^ ^TaCJa = Too 



where 

(3.3) 



a^\uP~N] 

Let P be a hyperbolic element of F* such that Tp = {12}- Assume that a is a 
fixed point of P. Then 00 is a fixed point of (j~^Po'a, and hence there exist positive 
integers a, d with ad = n,a ^ d such that 

(34) P=J-a r^ Ma-i = — r ^'^t^'''^]^ b[w\N]^ 

^'' y^ "^yo d) - ^y(a-d)^-b[w^N] d + b[w^N]^ 

Since c(P) depends only on the conjugacy class {P} represented by P, we can 
replace P by 'j~^P'j without changing the value of c{P). Replacing P by 'y~^P'j 

for some element 7 = cr^ ( j o""^ G F^, we can assume without loss of generality 

that ^ 6< |a-c?| in (3.4). 
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Lemma 3.4. Let a — u/w be given as in (3.1). Then 

^ I (a h\ _i 



with a,d E Ij.ad = n,a ^ d is a hyperbolic element of F* with Vp = {I2} and 
P(a) = a if and only if (w, N/w)\{a — d) with b being chosen so that P eT* . 

Proof. Let P be a hyperbolic element of F* with Fp = {I2} and -P(a) = a. By 
(3.4), there exists an integer k such that ^N\a — d — kw, and hence {WjN/w)\a — d. 
Conversely, if a,d are positive integers satisfying ad = n,a ^ d and (w, N/w)\a — 
d, then a — d = £{w, N/w) for some integer £. Integers A and r exist such that 
XaN + Tw = (ty, N/w), and hence £XaN = a — d — irw. Thus, an integer k exists 
such that —N\a — d — kw. Let A = a — kw, B = ku, C = —(a — d — kw) and 

D = d + kw. Then P = 4^(cd) ^^^^ element of F* with Fp = {I2} and 
P(a) = a, which is of the form 

1 fa b\ _i 



with a, d E 'L, ad = n, a ^ d and with b being chosen so that P G F*. D 

Lemma 3.5. Let a = u/w be given as in (3.1). Let a,d E Z+,a(i = n,a j^ d. If 

{w, N/w)\a~d, then there are exactly \a—d\ number ofro{N)-inequivalent hyperbolic 
elements P G F* with P{a) = a and Fp = {I2}, which are of the form 



jn 

with ^ 6 < \a — d\. 
Proof. For a, d E Z, ad = n, a j^ d, let 



1 ( a b \ _i 



a 



1 „ f^ b\ „-i _j D/_ 1 /a b'\^_i 



P = -^(Tn " , cr-^ and P' = -^a 



d "" ./^ ° V d 



n \ u a J wn 



a 



a 



be two elements of F* with ^ 6, 6' < |a — (i| whose fixed points are cusps. By 
using (3.4), we can find that b' = b + £ for some integer £. 

Conversely, for a given integer £ with ^ b + £ < \a — d\, if b' = b + £ then 

p/ 1 fa b'\ _i 

is a hyperbolic element of F* with Fp/ = {I2} and P'{o) = a. Moreover, if b' ^ b 
modulo I a — d\, then P' and P are not Fo(A^)-equivalent. Therefore, for fixed 
a,d E Z"'",a(i = n^a j^ d, there are exactly \a — d\ Fo(A^)-inequivalent hyperbolic 
elements P in F* with P{a) = a and Fp = {I2}, which are of the form 



1 f a b \ _i 



with ^ b < \a - d\. D 
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Lemma 3.6. Let a = u/w, and let 

D 1 /a 6\ _i , „, 1 (d b' 

be two hyperbolic elements of T* with -P(a) = a, P'(a) = a, Tp = {I2} and Tp/ = 
{I2}. Then P is Vq[N)- conjugate to P' for some number b' if and only if the two 
fixed points of P are Vq{N)- conjugate. 

Proof. Assume that the two fixed points of P are ro(A^)-conjugate. Then there 
exists an element T = \ ^) G ro(A'") such that a, T(a) are the two fixed points of 
P. Let P' = T-^PT. Then P'(a) = a and Vp, = {I2}. Write 



P' = -^cxa 



1 fa' b' 



cr„ 



-1 



v/?i^VO d' 
with a' d' G Z+, a'd' = n, and ^ b' < \a' — d'\. Since P' = T~^PT, we have 



^^«"'^^")(o d') = {o rf)(^^"'^^«^ 



From this identity, we can deduce that a' = d, and hence d' = a. 

Conversely, suppose that there exists an element T = ( j G ro(A^) such that 

T-^PT = P', that is. 

It follows that 

({a -d)-- 6[«;2, N]] u{A + B-) = b[w^, N]-{-Cu - Dw). 
\ u / u w 

This identity can be written as 

T(a) = ^— ^, 

^ ' -I a — a ' 

-■- b[w'2,N]a 

which is the second fixed point of P. That is, the two fixed points of P are Fq ( A^)- 
conjugate. D 

Lemma 3.7. Let a,d E Z+,a ^ d,ad = n, and let a = u/w be given as in (3.1) 
with {w,N/w)\{a — d). Assume that a and a' = ui/wi are the two distinct fixed 

point of P = -y^cTa [nfji'^a^- Then {w' , N/w')\{a — d) for w' = {wi, N), and 

p 1 fd b'\ _i 
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for some number h' . 

Proof. There exist positive integers a', d! with a'd' = n such that 

for some number b' , and hence we have 

(3.5) ^a-W(; ^') = (o rf)^a"'^«'- 

Write a-Va' = (cd)- ^^^^ ^^ ^^'^^ *^ S^* *^^* (^ " ^'^'^ = 0' (^' " ^)^ = ^'^ 
and {d — d')D = b'C. Since a 7^ 0', by using (3.3) we find that C 7^ 0, and hence we 
have a' = d. It follows that d' = a. By the argument made in the first paragraph 
of the proof of Lemma 3.4, we have {w' , N/w')\{a — d). D 

For a large positive number Y, define 

Dy = {z eD : '^a~^^z < Y, i = 1, 2, ■ ■ ■ , /i}. 

Let 

{Dp)y = U -fDy. 

-yeVoiN) 

Write 

c{P)y = / k{z,Pz)dz. 

J{Dp)y 

Lemma 3.8. Let a = u/w be given as in (3.1), and let P = -j^cTa ( n j ) ^^a^ ™^^ 

a,d E Z+, ad = n,a^ d be a hyperbolic element ofV* with Vp = {12} and -P(a) = a. 
Then 

\a- d\ d Ji \ n / V t - 1 

where o{l) ^0 as Y ^00 and p = C^/2[C^/f, N]Y with C = {a-d)^ -blw"^ ,N] 
andi = gcd{C,b[w^,N]^). 

Proof. Let 

p r 



7 

' q s 

be an element of SL2{M.). The linear fractional transformation, which takes every 
complex z in the upper half-plane into 'j{z), maps the horizontal line '<sz = Y into 
a circle of radius ^ \y with center at - + 2~W ■ ^^^ 

1 



^=lo V 



Then 



Note that 



Since 
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/ w/u b^[w2,N] bu^[w2,N] 

^a~^ = V^R^^ ^'^ { a-d)w 

1 _i (l/y 0\ fa 0\ fy \ _i 



for any y 7^ 0, by choosing y = {a — d) ^/^^^lP~N] we obtain that 

/■ a 

c{P)y = / A;(2, --jz)dz 

where 

_ /(a-rf)f -6[«;2,iV] 6[«;2^iV]^ 

Afo — I -1 u 



\ a — d {a — d)'w 

The hnear transformation z — * cTo^ maps the half-plane $52 > y into a disk Du 
of radius 2\w^^n]y ^^^^ center at a-|- 2\w'2^n]y - Then the transformation z -^ //a(-2) 
maps the disk Da into the half-plane '^z > {a — d)'^[w'^, N]Y. Let 

-b[w'^,N]u/w 



" ia-d)^-b[w^,Ny 
If 



fTB 



p/[cvF;iv] 



then (TpOO = p and cr^ Tp^-p = Too- By the definition of Dy, the image of the 
half-plane Qz > Y under the linear transformation ^ ^ cTp^ is not contained in 
{Dp)y- Since the linear transformation z -^ {^a'^p){z) maps the half-plane '^z > Y 
into a disk Dp of radius p centered at ip, where p = C^/2[C^/£^, N]Y, the disk Dp 
is not contained in pa{{Dp)Y}- It follows that 

. Aa-df[^^N]Y/ sine f^a-d)'\ dv 



'0 J2psin9 \nsm J rsm 

1 \ n J Vt^^ 

where o(l) has a limit zero when y ^ 00, and hence we have 

\a — d\ d Ji \ n / v ^ — 1 

This completes the proof of the lemma. D 
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Theorem 3.9. Let N he any positive integer, and let 

H = {(a; a, 6, d) : a, a, b, and d are given as in Lemma 3.4}- 
Then we have 

Y, c{P)Y = V^lnY Yl ^(1^^) 

{P},rp={l2} ^,ad=n,d>0,a^d 

(w,N/w) I (a — d) 



{a;a,b,d)eH 



- ja(i=n,d>0,a.^(i 

{w,N / -w) I (a — d) 

w;/iereC= (a - rf)f - 6[«;2, A^] an(i £ = ^c(i (C, 6[«;^ A^]^) . 

Proof. Let 5*1 be the set of conjugacy classes of hyperbolic elements P of F* such 
that Fp = {12} and such that the two distinct cusps of P are not Fo(A^)-equivalent, 
and let ^'2 be the set of conjugacy classes of hyperbolic elements P of F* such that 
Vp = {I2} and such that the two distinct cusps of P are Fo(A^)-equivalent. For the 
convenience of writing, we denote 



P(a;a,6,rf) = -^a,(;; ^) a^-^ 



where a = u/w is given as in (3.1). Since the two distinct cusps of every element P 
in Si are equivalent to Fo(A^)-inequivalent cusps of the form (3.1), we have 

E <^^^ = l E c{P{a;a,b,d))Y 

{P}eSi {a;a,b,d)eH 

P(a;a,b,d)eSi 

by Lemma 3.6. It follows from Lemma 3.6 and Lemma 3.7 that 
E <P^^ = l E c(P(a;a,6,rf))y. 



{P}eS2 {a;a,b,d)eH 

P(a;a,b,d)eS2 



It follows that 



Y c(P(a;a,6,rf))y = 2 Yl ^^^y- 

{a;a,b,d)eH {P},rp = {l2} 
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By Lemma 3.5 and Lemma 3.8, we have 

J2 c(P(a;a,6,rf))y = 2v/^lny J] g{ln^) 

{a;a,b,d)eH ^ ,ad=n,d>0,aj^d 

(w,N/w)](a-d) 

{a;a,b,d)eH 

+ 2^ \a-d\j k{- -t\ dt + o{l). D 

^,ad=n,d>0,a^d ^^ ^ "" / V t — 

(w,N/w)\(a-d) 

3.4. Parabolic components. 

Let S" be a parabolic element of F*. Then every element of ro(A^) having the 
same fixed point as S commutes with 5". If a = u/w is the fixed point of 5", then 
Fg = Fa, and hence we have a~^rso'a = Fqo, where aa is given as in (3.3). Since 
a~^Saa commutes with every element of Foo, we have 



^^"'^^" = ;^(o a) 



for some numbers a, b with a"^ = n, a & Z. This implies that F* has parabolic 
elements only if n is the square of an integer. Furthermore, by (3.4) we see that 
elements of the form 



S = (Ja 



{I "/f).,-'. o^iez 



constitute a complete set of representatives for the conjugacy classes of parabolic 
elements of F* having a as its fixed point. It follows that 



-Y rl 



y / k{z,Sz)dz = I I 2. k{z,z-\ — -=)dz + o{l), 
^s}Jd^ Jo Jo o_,^g^ V^ 

where the summation on {S} is taken over all parabolic classes represented by 
parabolic elements whose fixed point is a = u/w and where o(l) tends to zero as 
Y — > oo. Define 5^ to be one if n is the square of an integer and to be zero otherwise. 
The identity 

(A B\ _^_( Af--BaAw\N] Ba,a,[w\N] 

holds for di = Ui/w and Oj = Uj/w. 
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Lemma 3.10. If 

J_ (A B 

is an element of a~^T*aai, then A,D are integers satisfying {w, ^)\{A — D). Con- 
versely, if {w, ^)\{A — D), then there are exactly D elements in r,i.\r* represented 

by matrices of the form -^(Jai ( o d ) "^i"^' 

Proof. Assume that an element T = [" g ) G ro(A^) exists such that 

-irpfa b\ (A B 

^-^^1^0 rfj^«^=^0 D 

for some positive integers a, d with ad = n. Then 

_ {ab + /3d)w'^ 

Since 

1 fA B\ _i 

is an element of F*, A,D must be integers. By (3.6) we have that 

is an integer. Since {ui,w) = 1, it follows that 

B[w\N] _^, 

— = (modty). 

w 

By (3.6) we also have that N/w divides 

A-D B[w'^,N] 

Ui w 

It follows that 

A-D , ,, A^,, 
= (mod [yj, — )). 

Ui w 

Since (uijw) = 1, we have that 

TV 
A- D = (mod(w,—)). 

w 

Conversely, if (to, — )|(A — D), then it follows from the second part of the proof 

of Lemma 3.4 that 

1 fA B\ _i 

is an element of F* for some number of B. By an argument similar to that made 
in the proof of the lemma 3.5, there are exactly D elements in Fa^\F* represented 

by matrices of the form -4=0'a- i „ ,^ ) d,"^- 
This completes the proof of the lemma. D 
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The Maass-Selberg relation [7]. Let T be a discontinuous group of finite type, 
and let ai, ■ ■ ■ , a/i be a complete set of inequivalent cusps ofV. Assume that f{z, s) 
and g{z, s') are two automorphic functions with respect to V such that 

Af{z,s) = s{l-s)f{z,s) 

and 

^g{z,s') = s'{l - s')g{z,s') 

and such that the constant terms of the Fourier expansion at aj of f and g are given 
respectively by 



and 



cjv' + cy-' 



djy''+dy ''. 



Then, for the compact parts f^ and g of f and g, we have 



{f{z,s)r9'^{z,s')) 

^ fcjdjY'+''-^ - c'jd'jY-'-''+^ Cjd'^Y'-'' - c'^djY-'+'' 



s + s' — 1 s — s' 

j=l \ 

whenever s(l — s) 7^ s'(l — s') and f^ and g^ are square integrable on V\H. 
Theorem 3.11. We have 

u{N) 



J: [ i H Ej^-^,s)}E^{z,s)dz 

• 1 JDy „^-„ n^K^r} V " / 



= 1 "^ ^^ ad=n,Oi^b<d 



E E «'"'-'{- 



s + s- 1 



<'j = ^,i.^j,N/-Wj)\(a-d) 



1-S 



S (5 , o 

J = l a,d,b 

as Res ^ 1/2, where the sum on a,d,h has the same meaning as in (3.10) with 
A', D' being defined there and oyiX) tends to zero as Y -^ 00. 



Proof. Let 






ad^n^0^b<.d 



16 
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Then fi{z) is an automorphic function with respect to ro(A^). Let the constant term 



of the Fourier expansion at aj of fi{z) be Cijy'^ + c[,y^ ^. By the Maass-Selberg 



relation we have 

(3.7) 

y^ / ft{z)Ei{z,s)dz -oy{1 



\ 



ij^rjY-'^' 



s + s-1 



where oy(l) tends to zeros as Y ^ oo 

Since 

' aaa z + b 



Ei 



d 



7er,^\ro(Ar) 



_-, aaa.z + b 
y 1 -a. 7^^ 



y[^a, 7 1 n^]^a.Z 



we have 

McTa^z) = Yl Yl 

ad=n,0^b<d -yer „^\ro{N) 

Let a,d > and 6 be integers with ad = n and < 6 < (i. If the matrix equation 



a b 
d 



(3.8) 



0", 



■'l.T 



n 



a b 

rfJ^"^ 



J_fAB 
y^\0 D 



holds for an element T in ro(A^) and for some numbers A, D and B, then A, D are 
integers with AD = n. Moreover, by Lemma 3.10 we must have {wi, N/wi)\{A — D) 
for Oi = Ui/wi- Note that, for any fixed pair of positive integers A, D with AD = n, 
if the matrix equation (3.8) has a solution T, a, 6, d for a number S with < B < D, 
then the matrix equation (3.8) has solutions for exactly D number of S's with 
< B < D hj the proof of Lemma 3.5. By Theorem 2.7 of Iwaniec [6], we have the 
following decomposition 



r.V-'-i=r„(iv) (« ;;,<.,. 



*..^u u (; :)r. 



7>0 5 (mod 7) 



^[0 D 



* * 
7 5 



where the unions on 7, 5 run over all numbers 7>0, 0<5<7 such that ( 

belong to a~^ro {N)aa . Then it follows from Lemma 3.10 and the proof of Theorem 
3.4 of Iwaniec [6] that the constant coefficient in the Fourier expansion at infinity 
of the function 



fi{(^a,z) 



* 76r„Ar„(7V) 



n^ar7iodi^«.^ 
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where the sum ^^ runs over aU integers a, d > 0,0 < b < d, ad — n such that the 
matrix equation (3.8) has a solution T, A, S, D, is equal to 



(3.9) 



J2 {S^JAW'-^y^ + ^,,{s)A'-W^y'-^}. 



A,D>0,AD=n 



Next, we assume that a^d > and b are integers with ad = n and < b < d 
such that the matrix equation (3.8) has no solutions. Then the cusp {aaj + b)/d 
is not To (A^) -equivalent to aj. Because every cusp of ro(A^) is ro(A^)-equivalent to 
a cusp given in (3.1), a cusp a^', / 7^ j, exists which is equivalent to {aaj + b)/d 
under ro(A^). That is 



o-j' = 70 



aaj + b 
d 



for some element 70 in ro(A^). Let a a., be the element in PS'L2(M) such that 
0-0^,00 = aj' and a~^]raj,aaj, = Too, and let 



rr. -I I fa b\ 



Then Tq is an element of 5'L2(M) satisfying To(oo) = 00. It follows that we have 
the decomposition 



r„K'-l=r„(A^) ( 



a b 

d)""' 



U U 

7>0(5(mod7) 



* * 

7 d 



Tn 



* * 

7 s 



where the unions on 7, 5 run over all numbers 7>0,0<5<7 such that ( 

belong to a"~^ro(A^)o"a., . Note that, since fi(aajZ) and fi^aa^z) are both periodic 
function of x, ^ = x + iy, of period 1, the function 

is also a periodic function of x of period 1. Since To(oo) = 00, we can write 

^""T^lo D') 

for some real numbers A' , B' and D' . Then it follows the above decomposition, the 
above note, and the proof of Theorem 3.4 of Iwaniec [6] that the constant coefficient 
in the Fourier expansion at infinity of the function gi{aaz) is equal to 



(3.10) 



a,d,b 



1-s 
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where the sum is over aU integers a, rf > and < b < d with ad = n such that the 
matrix equation (3.8) has no solutions. It follows from (3.7), (3.9) and (3.10) that 

V / Mz)E,{z,s)dz-OY{l) 

i=l -^^^ 



I 



E E i- 



s + s-1 

i,3 = l A,D>0,AD=n 

{mj,N/vUj)\{A-D) 

s — s 

-> > \- \ ^ 1 

.^ ^, S + S-1 s-s 

i,j = l a,a,b 

where the sum over a, d, b has the same meaning as in (3.10). If $(s) is the constant 
term matrix of the Eisenstein series for ro(A^), then $(s) is a symmetric matrix 
and satisfies the identity 

$(s)$(l-s) = / 

for all complex s, where / is the identity matrix; see Kubota [7]. Since j ^ j', we 
have 

Yl ^ij'is)^ij{s) = 
1=1 

as Res -^ 1/2. Hence, the above identity can be written as 

i/(7V) uiN) 



J2 I Mz)E,{zrs)dz = J2 E 

i=l -^Dy ij^i A,D>0,A 



A'D 

AD=n 



s 7-il-s 



(3.11) .. 6,JY^+-^-'-V>^Jis)ipAs)Y-^-'+^ ^ ^^ ^,,{s)Y^-'-^,,is)Y-^+% 

s + s — 1 *'^ s — s 



1-s 






s — s 

i,j = l a,d,b 



as Re s -^ 1/2, where the sum on a, d, b has the same meaning as in (3.10). 
This completes the proof of the theorem. D 



Theorem 3.12. Put 



c{oo)y = Sni^{N) / / 22 ^{z^z~^ — -j=)dz — I H{z,z)dz. 
-^0 Jo o_,^g2; ^" J^^ 
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Then 

-^^^ / Hr)-{l + zr)dr+-h{0){6MN) 

j = l ad^n,d>0,a^d 

a-i = — ,{'uii,-^)\(a-d) 
J w^ 3 ' w j ' ^^ ' 



^y J 1/'°'^ / n \ ir 1 1 



.(TV) 

^ iyr / 

i,j=l ad=n,d>0 "^ °° 

1 ^^^^ * //4'\^/^ 1 

3 = 1 a,d,b ^ ^ 

where oy (1) tends to zero as Y ^' oo, and t/ie snm on a, d, 6 /las the same meaning 
as in (3.10) with A' , D' being defined there. 

Proof. By the argument of [7], pp. 102-106 we have 

• F rl 



V^io Jo 0^, V^' 

= ^(0)ln(v^y) / h{r) — {l + ir)dr-g{0)ln2 + -h{0)+o{l). 

27r J_^ r 4 



Let 



c ' ' \ d / 



where the sum runs over c > 0, (i modulo c with ( ^ ^) G (7jj,^ro(A^)craj. . Then we 

have 

E,{aa^z, s) = dijv'' + y?ij(s)y^"^ 

+ -FtX- E \M'~^K^_^{2\m\'Ky)^ij^^{s)e{mx) 
where 

'^'^'^^'^ = f(T\ V^J,0(S)• 
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By Theorem 3.11, we have 



i=l •" ^Y ad=n,0^b<d 



^ ^ """^ ^ s + s-l 



aj = ^,{wj,N/-Wj)\(a-d) 



+ *' J^i }- Z. 2. J— + "v(i) 

i,i = l a,d,6 



as Re s -^ 1/2, and hence we have 



E/ i E i?.(^,«)}i?.(^,«-)c/. 

i=l "'^y ad=n,0^&<(i ^ ^ 

^ ^ S + S- 1 

j = l a,d>0,ad=n 






\-S 



+ T^i } " ^ ^ T^i + '"' ' 

J = l a,d,b 



as Res -^ 1/2, where the sum on u/w runs over aU cusps given in (3.1) and the 
sum on a,d,b has the same meaning as in (3.10) with A',D' being defined there 
and oy(l) tends to zero as F — *> cxd. 

By partial integration, we obtain 



1 r^ 

Hr) = - g^^'Hln 

f Jo 



u)u'''^ du 
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for nonzero r. Then it foUows that 
(3.12) 

uiN) 



lim^ /" h{r){J2 [ I Yl E,{^^^^,S + ir)\ E,{z,S-ir)dz}dr 
= v^E E {47r<7(ln-)lny-5^ / %) h) V^:,(2 + ^^ 



j—1 ad—n,d>0 i—1 






X ^^^.(1 _ ir)rfr + r h{r) (^)" ^2^k_!!0^^^^ 

EE/ Mr)¥^,y(2+^0(;^ — — c^r + oy(l). 



j = l a,d,b ' 



By the Riemann-Lebesgue theorem (cf. §1.8 of [16]), we have 



(3.13) "-"■'- ^"^ 



By (3.12) and (3.13), we have 

-= H{z,z)dz=j2 E {9{in^)^^y + ^m^n{^) 

a T — — — Aw A , -^^ )\a — d 
J -w j • J ^ -w j ' ^ 

- 2^ 4^ y_^ ^(^) \2) ^''^2 ^ '''^^'^^2 " '""^ ^^ 

J = l a,d,b ^ ^ 



The stated identity then foUows. D 
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It follows from Theorem 3.9 and Theorem 3.12 that 

(3.14) 

lim ( c(oo)y + y^ c{P)y 
^^°° \ {P},rp={i2} 



r r "^^^ 1 



j = l ad^n,d>0,a=^d 

a^=-^,(u),-,-^)|{a-d) 



it)|Af,K;>Oad=n,d>0,a/(i b ' ' 



dniy{N)^ 



/OO p/ 

/i(r) — (1 + ir)dr 
-OO -*- 



27r 






j = l ad^n,d>0 ^ ^ 2=1 






+ V -\a-d\l k\— —t] , ^ -At 



ad=n,d>0,a=^d 

>., JV_)|(a- 

i^(^') * / ./N 1/2 



"i = ^-('"i-lf^)l(''-'^) 



1 /^^ * /A'\ 

7 = 1 aA,b ^ ' 



^Jj'ilj) 



with C = {a — d)w — bwN, where i = (C, bN) and the summation on b is taken over 
all numbers b such that C, bN/w G Z, A^|C and ^ 6 < |a — (i|. Note that there are 
exactly |a — (i| number of such numbers b by Lemma 3.5. 

Denote by c(oo) the right side of the identity (3.14). We conclude that the trace 
formula (2.3) can be written as 



t 



d{n)h{--) + ^/n'^h{KJ)trjTr, 
(3.15) '=^ 

= ciI)+J2<R)+ E c(P) + c(oo) 
{R} {P},rp^{u} 

for Re s > 1 , where the summations on the right side of the identity are taken over 
the conjugacy classes. 

Theorem 3.13. Let c{P) = Jp >^ k{z, Pz)dz for hyperbolic elements P G ro(A^). 
Then the series 

E^(^) 
{p} 
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represents an analytic function in the half-plane Res > except for having a possible 
pole at s = 1/2 and for having simple poles ai s = 1, | ± ikj, j = 1, 2, ■ ■ ■ . 

Proof. We have 

g^^\\ogu) = A{s)ui-' + 0,(w-^), 

where A{s) is an analytic function of s for Res > and, for every complex number 
s with Res > 0, there exists a finite constant B{s) depending only on s such that 

\Os{u~^)\ ^B{s)u-^. 

Moreover, for every fixed value of u, the term Os{u~2) also represents an analytic 
function of s for Re s > 0. Since 



h{r) = — / g^^\lnu)u'''-^du 
f Jo 



for nonzero r, we have 



r^ \s—-^—tr s — ^ + ir / 

for Res > 1 and for nonzero r with |Imr| < i — e. By analytic continuation, we 
obtain that 

(3-16) Hr)= !.f'%j\, +0^{r-^) 

for Res > and for nonzero r with |Imr| < | — e. It follows from results of [17] 
and (2.3) that the left side of (3.15) is an analytic function of s for Res > except 
for having simple poles at s = 1, | ± zkj, j = 1,2,---. Then the right side of 
(3.15) can be interpreted as an analytic function of s in the same region by analytic 
continuation. 

Since k{t) = (1 +t/4)~^, by Lemma 3.1 we have that c{R) is analytic for Re s > 
except for a simple pole at s = 1/2. There are only a finite number of elliptic 
conjugacy classes {R}- The term c(/) is a constant. 

Since g{0) = 2^T{s - ^)T{s)-\ 



h{0) = 2V^r\^ / {U + - + 2) 
T{s) Ji u 



U 



and 



,, a, ;— ,„ 1 r(s — ^) fa d 
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the sum of first three terms and the last two terms on the right side of the identity 
(3.14) is analytic for Res > except for a pole at s = 1/2. 
By Stirling's formula the identity 

(3.17) ^^ = lnz + 0{l) 

holds uniformly when | arg^| < n — 6 for a small positive number S. It follows from 
(3.13) and (3.17) that the fourth term on the right side of the identity (3.14) is 
analytic for Res > except for a possible pole at s = 1/2. 

By Theorem 4.4.1 of Kubota [7], each Eisenstein series Ei{z, s) has a meromor- 
phic continuation to the whole s-plane, and the identity 



J2 V'^j f 2 + ^^) ^'^ (2 ~ ^^ ] = 1 



hold for all real r and for i = 1,2, ■•■ ,i/{N). It follows that functions ipij{s), 
i,j = 1,2, • ■ ■ , i'{N), are analytic on the line Res = 1/2. Let y be a fixed large 
positive number. By the Maass-Selberg relation, we have 

^^ 2^ V'^J ( 2 ^ ^^ J ^'^ V 2 ~ ^^ 

(3.18) -21nr I V^..(l/2-zr)y^--y:,,(l/2 + zr)y-^- 

2zr 

" Jo ^' V' ^ ^ V ^' v' I ~ V '^^ ^ ''^^^' 

Let Oi = Ui/wi be a cusp given in (3.1), and let rj = 1/2 + ir. By the argument 
following Lemma 3.6 of [3], we have 

E,{Z, 1]) = da^ooy"^ H p. . ^ '^a^ooMV^'"^ 

(3-19) 271"^ /y r-^ 1 

+ p/ . 2I l"^r"'^7,-i(2|m|7ry)(/7fl^oo,m(??)e(mx) 
where 
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and 

[{wi,N/wi)\'^ >r^ 1 >r^ [ md 

{c,N/wi) — l dYnod(cwi), (d,cwi) — l 

It follows from the functional identity of the Riemann zeta-function that 






X 



Hrri^nd-^'- 



WjA^ / -^-'- 1 — p 



p\N ^ -1^ 



By using Stirling's formula 

\T{a + it)\ ~ v^e-"l*l/2|t|'^-i/2 
for any fixed real value of cr as t ^ oo, we obtain that 

(3.20) ^^|!~^^V a.oo,o(r7)y^-^ « v^ln^drl + 1)). 

By (1.1) and by partial integration, we find that 

.3 2l^ Kr,-i/2{2\m\7ry) ^ 2^-^^ f^ cos{2\m\nyt) ^^ ^ 1 + \r\^ 

^ ' '' "r'/,^\ /ol — I \n — 1 /9 /ZZ I /I I -i-'?.\7i 



r{T]) (2|m|7ry)''-i/20f y^ (l + t2)^ \m\^y^' 

Then it follows from (3.16), (3.18)-(3.21), Lemma 4.7 of [3] together with the proof 
of Theorem 1 in [8], and Cauchy's inequality that the fifth term on the right side of 
the identity (3.14) is analytic for Res > except for a possible pole at s = 1/2. 

Also, it follows from (3.12) and Lemma 8.2 of Strombergsson [15] that the fifth 
term on the right side of the identity (3.14) is analytic for Res > except for a 
possible pole at s = 1/2. 

Therefore, by (3.15) we have proved that the series 

{p} 

represents an analytic function of s in the half-plane Res > except for having 
a possible pole at s = 1/2 and for having simple poles at s = 1, ^ ± mj, j = 
1,2,.... D 
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4. Proof of the Main Theorem 

Proofs for the following lemmas from Lemma 4.1 to Lemma 4.4 are minor modi- 
fications of corresponding lemmas in Conrey-Li [2] . For the convenience of readers, 
we sketch the proofs here. 

Lemma 4.1. Let X > be an eigenvalue of A for To{N) with N square free. 
Assume that (n, A^) = 1. Put r = 1/2 + in with k = ^JX — 1/4. Then we have 

A^V^^^trTr, = \im{s - r) V c(P) 
^ ' {n,rp/{i2} 

where the right side is defined as in Theorem 3.13. 
Proof. By (2.2), (3.13) and results of [17], we have 

oo 

lim(s — r) y h{Kj)trjTn = 0. 
By the proof of Theorem 3.13, we have 



lim(s - r) d{n)h{ — ) - c(/) - c(oo) - V c{R) 

S^T \ 2 ^-^ 

{R} 



0. 



Then the stated identity follows from (2.2), (3.12) and Theorem 3.13. D 

Remark. Let hd be the number of inequivalent classes of primitive indefinite qua- 
dratic forms of discriminant d. Siegel [14] proved that 

(4.1) hm M^^]^ = 1. 

d^oo Ind 2 

Lemma 4.2. P is a hyperbolic element of T* with Tp ^ {I2} if and only if there 
exists a primitive indefinite quadratic form [a, 6, c] of discriminant d such that 

v^ [aNu{a,N)-^ "+''^"("'^)" 

with 

v^ - {dN^{a, N)-'^}u'^ = 4n. 

If Xp is an eigenvalue of P , then 

1 _ Nu Vd 
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Let 

where the pair {vq, uq) is the fundamental solution of Pell's equation v"^ — dv^ = 4. 
Then P is Vq{N)- conjugate to a hyperbolic element P' G F* with Vpi 7^ {I2} if and 
only if Pq is Vq{N)- conjugate to Pq, where Pq is associated with P' similarly as Pq 
is associated with P. 

Proof. Let 

P=^(^ ^ 

be a hyperbolic element of F* such that Tp 7^ {12}- Then fixed points ri, r2 of 
P are not rational numbers, and hence Tp is the subgroup of elements in Fo(A^) 
having ri, r2 as fixed points. Let a = C//U, b = (D — A)/fx and c = —B/n, where 
// = (C, D — A, —B). Then [a, 6, c] is a primitive quadratic form with ri, r2 being 
the roots of the equation ar"^ + 6r + c = 0. By Sarnak [10] , the subgroup of elements 
in 5'L2(Z) having ri, r2 as fixed points is generated by the primitive hyperbolic 
element 

(--^^ -cuo 

where the pair {vq, uq) is the fundamental solution of Pell's equation v"^ — dv? = 4. 
Since P and Pq have the same fixed points, we have A = D — bC/a and B = 
—cC/a. Since P belongs to F* and AD — BC = n, C satisfies 

( aD^- bDC + cC^ = na 

^ '^^ t a\C, N\C. 

Let Xp be an eigenvalue of P. Then it is a solution of the equation A^ y£-X+l = 0. 

By using A = D — bC/a and B = —cC/a, we obtain that 

(4.3) ^- f - ±^ 

Ap a^n 

and 

(4.4) Ap + -3- = ^(2I)--C). 

Ap Jn a 



Conversely, let a pair (C,D) be a solution of the equation (4.2). Define A 
D — bC/a and B — —cC/a. Then the matrix 

^{C D 
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is a hyperbolic element of F* with Fp ^ {I2} having the same fixed points as Pq, 
and eigenvalues of P satisfies (4.3) and (4.4). 

Next, let V = 2D — b— and ui = —. Then the equation (4.2) becomes v'^ — du\ = 
An with N\aui. Since N\aui, this equation can be written as 

(4.5) v^ - - — ^TTT^^u^ = 4n. 



{a,Ny 



Moreover, we have 



.^g. J_fA B\ _ 1 f -^^<-^^r' ^cNu{a,Ny 



n\C D) ^\aNu{a,N)-^ v+hNu{a,Ny 

Let Pq be the primitive hyperbolic element of SL2{Z) corresponding to [a' , b', c']. 
Two forms [a, 6, c] and [a', b' , c'] of the same discriminant are equivalent in ro(iV) if 
and only if an element 7 G ro(A^) exists such that 7~^-Po7 = Pq- D 

Lemma 4.3. Let [a, 6, c] 6e a primitive indefinite quadratic form of discriminant 
d, and let 

V^\aNu{a,N)-^ -+'^^(-^^y 

with v^ — {dN^{a, N)~^}u'^ = An he a hyperbolic element ofT* with Tp 7^ {12}- Let 
{vi, ui) with vi,ui > be the fundamental solution of Pell's equation v'^ — diu'^ = A, 
where di = dN'^/{a,N)'^ . Then Tp is generated by the hyperbolic element 

1 f„. U Nui 



p^_ . 2h-^KiV) 



a^^^ 




1 (v, I h ^"1 

■{a,N) 2 y^ ^" {a,N) 

ofToiN). 

Proof. Let a, b and c be given as in the proof of Lemma 4.2. Since Fp is cyclic, a 
solution f 1, u'l > of Pell's equation v^ — du'^ = A exists such that Fp is generated 
by 

aui 

and hence Pi is the smallest positive integer power of Pq among all powers of Pq 
belonging to Fo(A^). Since N\au'i, we have , '^x \u'i. Write 




•Ui 



{a,N) 

Nui 



' (a, N) ■ 

Then the pair (vi^ui) with vi,ui > is the fundamental solution of the Pell equa- 
tion v"^ — diu^ = 4, where di = dN'^{a, iV)~^. The stated result then follows. D 

Two quadratic forms [a, 6, c] and [a',6',c'] are equivalent under Fo(A^) if an ele- 
ment 7 G Fo(A^) exists such that 

a' b'/2\_,(a b/2\ 
b'/2 c' ~^ U/2 c p- 
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Lemma 4.4. Let [a^, hj, Cj], j = 1, 2, ■ ■ ■ , Hd, be a set of representatives for classes 
of primitive indefinite quadratic forms of discriminant d, which are not equivalent 
under ro(A^). Then we have 

{p}: P6r*,rp^{i2} 

-r(s-i)^^^(a„iV)lned, / , d{Nuf 



nn- 






for ^s > 1, where di = dN'^{aj, N) ^ and the sum on u is over all the positive 
integers u such that An + dN'^{aj, N)~'^v?' is the square of an integer. 

Proof. It follows from (2.1) and Lemma 3.2 that 

^ '^^^ = '^^X^ ^ Ap - 1/Ap [^ + 4 

for Ks > 1, where Ap > 1 is an eigenvalue of P and Pi is given in Lemma 4.3. Let 
P be associated with a primitive indefinite quadratic form [aj,bj,Cj] as in Lemma 
4.2. Then by Lemma 4.2, we have 

_J__ Nu Vd 
Xp {aj,N)y/n' 
By Lemma 4.3, we have 

V^^i = ^ = fdi- 

If P' is a hyperbolic of F* with Tpi ^ {^2}-, and is associated with a primitive 
indefinite quadratic form [a^/, 6j/, Cj/] as in Lemma 4.2, then P and P' are ro(A^)- 
conjugate if and only if [%•, 6j, Cj] and [aji ,hji ,Cji] are ro(A^)-conjugate. Let T be 
a hyperbolic of F* with Fp 7^ {I2} associated with a primitive indefinite quadratic 
form [a, 6, c] as in Lemma 4.2. If the discriminant of [a, 6, c] is not equal to the 
discriminant of [a^, hj, Cj\ which is associated with P, then P and T are not Fo(A^)- 
conjugate. The stated identity then follows. D 

The proof for the following lemma is a minor modification of a corresponding 
lemma in Conrey-Li [2], and is given in Cardon-Li [1]. 

Lemma 4.5. Let k he a divisor of N . Then the number of indefinite primitive 
quadratic forms [a, 6, c] with (a, A^) = k of discriminant d, which are not equivalent 
under Fo(A^), is equal to 

V n '''■n(i+(f 

p2i|(d,fc) p\k ^ ^^ 

where di = dN'^/k'^ , where the first product runs over all distinct primes p with p^^ 
being the greatest even p-power factor of {d, k) and the second product is taken over 
all distinct primes p dividing k. 
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Theorem 4.6. Let N he a non-square free positive integer with {n,N) — 1. Then 
we have 



J2 c{P) = 4v^-^^' ^^ 



T(s) 
{p}: Per*,rp/{i2} "■ ^ 

n{n) \^ \-^ Yj if d ^ hdlTa.ed f^ ^ dh'^u" 



/or ^s > 1, where the sum on u is over all the positive integers u such that 
V^nTdkhJ? e Z. 

Proof. Let [aj,6j,Cj], j = 1,2,- ■■ , -ff^, be a set of representatives for classes of 
primitive indefinite quadratic forms of discriminant d, which are not equivalent 
under ro(A^). By Lemma 4.4, we have 

.r(s-i) — ^— 1"- / ^..,2\i 

'' T(s) 
{P}: Per*,rp^{i2} ^^ denj=i u 

for lR,s > 1, where di = dN'^{aj, N)~'^ and the sum on u is over all the positive 
integers u such that ^/4n + diu^ G Z. Denote k = N/{aj, N). By Lemma 4.5, we 
can write the above identity as 

J2 c(P) = 4V^^^ 



-£-.../ ^ r(.) z^z^z.^^^^ 4^; 



X 



for Ks > 1, where (ii = dk"^. By Dirichlet's class number formula 

/id, Ined, = A/rf^L(l,XdJ 
and the identity 

p\k ^ ^^^ 
we can write (4.6) as 

{p}: PeT*,Tp^{i2} ^ ' k\Nden u p2i\{d,f) 



f ' '^ ' ' P\k 
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for ^s > 1, where the sum on u is over all the positive integers u such that 

\/4n+~dkHt^ E Z. Since 



Pif 



n i+n^ n 1 



p 



p\k 



P 



EE 



d \ /u(n) 



mn J n 



we have 



E 



c(P) = Ay/jm 



(4.7) 



{p}: Per*,rp^{i2} 



Vis) 



X 



^(n) 



^ ^ n ^^ ^^,, \mn) uVd 



den u p2!|(-rf W) 
Next, we want to show that 



2^,2\ 2 



An 



d 



Z-^ Z-^ \ mn I J-J- 



den u - / p^^\{d,f) "^" 



ihdlned A , dk^u 
P — I 1 + 



2,,2\ 2 



4n 



is absolutely convergent for a = ^s > 1. Since 



(4.8) 



EE n ^' 

den u p2i\(d,f) 



mn) uVd 



1 + 



dk^u 



2^,2 \ 2 



4n 



/ — \-^ v^ hd In ew / dk'^v? ^ ^ 



dgfi li 



W 



V^ 



An 



It is proved in Li [9] that the right side of (4.8) is convergent for a > 1, and hence, 
the right side of the stated identity is absolutely convergent for 3f?s > 1. 
This completes the proof of the theorem. D 



Proof of Theorem 1. It is proved at the end of Li [9] that 



E 

den,u 



hdlned,. dv? ,1 „ , , -^„ i hdl'tied 
Vdu 4n 



{du 



2W 



«E 

den,u 



{du 



2^|i+e-l-o- 



U 



l+2e 



< OO 



for a > 0. Then it follows from Theorem 4.6 that 

(4.9) 

1 . ^ 4v/^r(z^)(4n)- 

lim (s IK] > ciP) = 



{P},rp/{i2} 



r(l/2 + m) 



X 



^(n) 



1 



E E E ^^^'^"'^'^ hm . (.-.-z^) 



k\N m\^ n\k 



n 



s^l/2+iK 



EE n p' 

den u p2^(^d,f) 



d \ hdlned 



mn J {du 



2^s 



32 XIAN-JIN LI 

Theorem 3.13 shows that the function on the right side of (4.9) represents an analytic 
function in the half-plane 3f?s > except for a possible pole at s = 1/2 and for 
possible simple poles at s = 1, ^ ± iKj, j = 1,2,---. The stated identity then 
follows from Lemma 4.1. 

This completes the proof of the theorem. 
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